Range-gated depth camera assembly

ABSTRACT

An augmented reality (AR) includes a depth camera assembly (DCA) to capture images of various depth zones of scenes of a local area. The DCA can focus on specific ranges in a scene, important aspects, and/or regions of interest. The DCA generates image data of the local area such that the image includes information pertaining to a single depth zone. The captured image is specific to the single depth zone and is representative of objects within the single depth zone. The DCA uses the generated image data for the depth zones to generate augmented or partially-augmented images that include depth information for the objects in the local area.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 62/268,488, titled “Range-gated Depth CameraAssembly,” filed on Dec. 16, 2015, which is incorporated by referenceherein in its entirety.

BACKGROUND

The present disclosure generally relates to depth camera architecturesand more specifically relates to a range-gated depth camera assembly.

Virtual reality (VR) systems, which may include augmented reality (AR)systems, can leverage the capture of the environment surrounding a userin three dimensions (3D). However, traditional depth cameraarchitectures are comparably large in size, heavy, and consumesignificant amounts of power. Moreover, different depth cameraarchitectures (e.g., time-of-flight, structured light, and stereovision) generally perform best in different operating environments, asthey each have different strengths/weaknesses. Additionally, traditionaldepth camera architectures are not suited to expanded operational ranges(centimeters to multiple meters) found in VR system. For example, suchexpanded operational ranges may result in too much data for traditionaldepth camera architectures to efficiently process, and may possessdynamic ranges that are too large for traditional depth cameraarchitectures to handle.

SUMMARY

A headset include in an augmented reality (AR) system or in a virtualreality (VR) system includes a depth camera assembly (DCA) enablingcapture of visual information of various depth zones of a scene of anarea surrounding the headset and within a field of view of an imagingdevice included in the headset (i.e., a “local area”). The DCA can focuson specific ranges in a scene, important aspects, and/or regions ofinterest. The DCA includes one or more imaging devices, an illuminationsource, and a controller. The illumination source emits light pulses(e.g., infrared (IR), visible) used to create augmented images. Theimaging device records light over an exposure time period to generateimage data.

The DCA illuminates a scene of a local area with one or more lightpulses. Objects in the local area reflect light (i.e., light pulses)emitted by the DCA. The DCA is configured to generate image data of thelocal area such that the image includes information pertaining to asingle depth zone (e.g., by controlling a time between emitting a lightpulse and image capture). The captured image is specific to the singledepth zone and is representative of objects within the single depthzone. The DCA generates image data for each of the depth zones. The DCAuses the generated image data for the depth zones to generate augmentedor partially-augmented images. An augmented image is an image of thelocal area that includes depth information (i.e., distance from anobject to the DCA) for one or more objects in the local area. Apartially augmented image includes image data for some, but not all, ofthe depth zones in a local area.

Note, in some embodiments, multiple pulses are emitted for a given depthzone before reading out image data associated with the given depth zone.This improves a signal strength associated with image data in the givendepth zone, as light from successive pulses are added. The DCA can thenread out image data that was generated using a plurality of pulses(e.g., 100 pulses as an example). The improved signal strength is usefulin capturing accurate image data for objects with low reflectivityand/or objects that are distant from the DCA. Likewise, the DCA can setthe number of pulses to control image saturation.

In one embodiment, a range-gated depth camera assembly includes anillumination source configured to emit illumination light pulses and animaging device configured to generate image data of a local area. Therange-gated depth camera assembly further includes a controller. Thecontroller is configured to regulate the illumination source to emit aplurality of illumination light pulses to illuminate a scene of a localarea. The local area is divided into a plurality of depth zonesincluding at least one non-empty depth zone that includes at least oneobject an imaging device configured to generate image data of a localarea. The controller is configured to, for each non-empty depth zone ofthe plurality of depth zones, regulate the imaging device to record aset of reflected light pulses to generate image data for the non-emptydepth zone. The set of reflected light pulses are the illumination lightpulses reflected off one or more objects included in the non-empty depthzone. The controller is further configured to generate an image of thescene using the image data generated for the at least one non-emptydepth zone of the plurality of non-empty depth zones, and provide theimage of the scene to a console.

In one embodiment, a head-mounted display (HMD) includes an electronicdisplay and a range-gated depth camera assembly that includes anillumination source configured to emit illumination light pulses and animaging device configured to generate image data of a local area. TheHMD further includes a controller configured to includes a controller.The controller is configured to regulate the illumination source to emita plurality of illumination light pulses to illuminate a scene of alocal area. The local area is divided into a plurality of depth zonesincluding at least one non-empty depth zone that includes at least oneobject an imaging device configured to generate image data of a localarea. The controller is configured to, for each non-empty depth zone ofthe plurality of depth zones, regulate the imaging device to record aset of reflected light pulses to generate image data for the non-emptydepth zone. The set of reflected light pulses are the illumination lightpulses reflected off one or more objects included in the non-empty depthzone. The controller is further configured to generate an image of thescene using the image data generated for the at least one non-emptydepth zone of the plurality of non-empty depth zones, and provide theimage of the scene to a console.

In one embodiment, a method includes emitting, by an illuminationsource, a plurality of illumination light pulses to illuminate a sceneof a local area. The local area is divided into a plurality of depthzones comprising at least one non-empty depth zone. The non-empty depthzone includes at least one object. The method further includes, for eachdepth zone of the plurality of depth zones, recording a set of reflectedlight pulses to generate image data for the non-empty depth zone. Theset of reflected light pulses is the illumination light pulses reflectedoff one or more objects included in the non-empty depth zone. The methodfurther includes generating an image of the scene using the image datagenerated for the at least one non-empty depth zone of the plurality ofnon-empty depth zones. The method further includes providing the imageof the scene to a console.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system environment including an augmentedreality system, in accordance with an embodiment.

FIG. 2 is a diagram of an augmented reality headset, in accordance withan embodiment.

FIG. 3 is a cross section of a front rigid body of a augmented realityheadset, in accordance with an embodiment.

FIG. 4 illustrates an example DCA capturing image data of a sceneincluding different objects at different distances, in accordance withan embodiment.

FIG. 5 illustrates a flow diagram of a process of generating augmentedimages, in accordance with an embodiment.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles, or benefits touted, of the disclosure described herein.

DETAILED DESCRIPTION

System Overview

FIG. 1 is a block diagram of one embodiment of an augmented reality (AR)system environment 100 in which an AR console 110 operates. As usedherein, an AR system environment 100 may also include virtual realitysystem environments that present users with virtual environments withwhich the user may interact. The AR system environment 100 shown by FIG.1 comprises an AR headset 105 and an AR input/output (I/O) interface 115that are each coupled to an AR console 110. While FIG. 1 shows anexample system 100 including one AR headset 105 and one AR I/O interface115, in other embodiments any number of these components may be includedin the AR system environment 100. For example, there may be multiple ARheadsets 105 each having an associated AR I/O interface 115, with eachAR headset 105 and AR I/O interface 115 communicating with the ARconsole 110. In alternative configurations, different and/or additionalcomponents may be included in the AR system environment 100.Additionally, functionality described in conjunction with one or more ofthe components shown in FIG. 1 may be distributed among the componentsin a different manner than described in conjunction with FIG. 1 in someembodiments. For example, some or all of the functionality of the ARconsole 110 is provided by the AR headset 105.

The AR headset 105 is a head-mounted display that presents content to auser comprising augmented views of a physical, real-world environmentwith computer-generated elements (e.g., two dimensional (2D) or threedimensional (3D) images, 2D or 3D video, sound, etc.). In someembodiments, the presented content includes audio that is presented viaan external device (e.g., speakers and/or headphones) that receivesaudio information from the AR headset 105, the AR console 110, or both,and presents audio data based on the audio information. An embodiment ofthe AR headset 105 is further described below in conjunction with FIGS.2 and 3. The AR headset 105 may comprise one or more rigid bodies, whichmay be rigidly or non-rigidly coupled to each other together. A rigidcoupling between rigid bodies causes the coupled rigid bodies to act asa single rigid entity. In contrast, a non-rigid coupling between rigidbodies allows the rigid bodies to move relative to each other.

The AR headset 105 includes a DCA 120, an electronic display 125, anoptics block 130, one or more position sensors 135, and an inertialmeasurement Unit (IMU) 140. Some embodiments of the AR headset 105 havedifferent components than those described in conjunction with FIG. 1.Additionally, the functionality provided by various components describedin conjunction with FIG. 1 may be differently distributed among thecomponents of the AR headset 105 in other embodiments.

The DCA 120 is configured to capture one or more images of an areaproximate to the AR headset 105, also referred to as a “local area,”using the one or more imaging devices included in the DCA 120. Someembodiments of the DCA 120 include one or more imaging devices (e.g., acamera, a video camera), an illumination source, and a controller. Theillumination source emits light pulses (e.g., infrared (IR), visible)used to create augmented images. For example, the illumination sourceemits a series of light pulses for a predetermined time period. In someembodiments, the illumination source may be a structured light (SL)illuminator that is configured to emit a SL pattern, such as a symmetricor quasi-random dot pattern, grid, or horizontal bars, onto a scene.

The DCA 120 captures images of the local area using one or more imagingdevices. The imaging devices capture and record particular ranges ofwavelengths of light (i.e., “bands” of light). Example bands of lightcaptured by an imaging device include: a visible band (˜380 nm to 750nm), an infrared (IR) band (˜750 nm to 1500 nm), an ultraviolet band (10nm to 380 nm), another portion of the electromagnetic spectrum, or somecombination thereof. The one or more imaging devices may also besensitive to light having visible wavelengths as well as light having IRwavelengths or wavelengths in other portions of the electromagneticspectrum. For example, the imaging devices are red, green, blue, IR(RGBI) cameras. In some embodiments, the one or more imaging devicescomprise a charge coupled device (CCD), a complementarymetal-oxide-semiconductor (CMOS) imager, other light sensitive device,or some combination thereof.

The DCA 120 may vary different characteristics of emitted light pulses.For example, the DCA 120 may vary, e.g., a pulse width, a pulseamplitude, a time between pulses, a quantity of pulses to emit, a pulsewavelength, or some combination thereof. As discussed in detail belowthe DCA 120 may vary pulse characteristics (e.g., pulse amplitude,quantity of pulses, etc.) to prevent saturation of a captured image andto ensure image quality of the captured image. Additionally, the DCA 120regulates when image data is collected by controlling e.g., an exposuretime period to record light to generate image data and/or a start timefor an exposure time period (relative to when a pulse was emitted by theDCA 120).

The local area may include objects at different distances from the VRheadset 105. In some embodiments, the DCA 120 scans the local area toidentify objects included in the local area. The DCA 120 then determinesa distance between each identified object and the DCA 120. The DCA 120divides the local area into one or more depth zones based on thedetermined distances to the identified objects. Alternatively, the DCA120 divides the local area into a predetermined number of depth zones.

The DCA 120 captures image data of the depth zone(s) one at a time. TheDCA 120 controls when a light pulse is emitted relative to when imagedata is to be collected. Accordingly, the DCA 120 is able to selectivelycollect image data specific to a depth zone. For a particular depthzone, the DCA 120 illuminates the depth zone (e.g., by a light pulse)and captures light being reflected off objects included in the depthzone. The DCA 120 uses the captured images of one or more depth zones togenerate an augmented image. An augmented image is an image of the localarea that includes depth information (i.e., distance from an object tothe DCA) for one or more objects in the local area. DCA 120 is furtherdescribed below in conjunction with FIGS. 3-5.

The electronic display 125 displays 2D or 3D images to the user inaccordance with data received from the AR console 110. In variousembodiments, the electronic display 125 comprises a single electronicdisplay or multiple electronic displays (e.g., a display for each eye ofa user). Examples of the electronic display 125 include: a liquidcrystal display (LCD), an organic light emitting diode (OLED) display,an active-matrix organic light-emitting diode display (AMOLED), someother display, or some combination thereof. In some embodiments,portions (e.g., a front side) of the AR headset 105 are transparent tovisible light, to allow a user of the headset to view the local areathrough the AR headset 105. In these embodiments, the electronic display125 is made up of one or more transparent electronic display panels. Atransparent electronic display panel is partially or fully transparentand may be, for example, a transparent organic light emitting diodedisplay (TOLED), some other transparent electronic display, or somecombination thereof.

The optics block 130 magnifies image light received from the electronicdisplay 125, corrects optical errors associated with the image light,and presents the corrected image light to a user of the AR headset 105.In various embodiments, the optics block 130 includes one or moreoptical elements. Example optical elements included in the optics block130 include: an aperture, a Fresnel lens, a convex lens, a concave lens,a filter, a mirror element, or any other suitable optical element thataffects image light. Moreover, the optics block 130 may includecombinations of different optical elements. In some embodiments, one ormore of the optical elements in the optics block 130 may have one ormore coatings, such as anti-reflective coatings.

Magnification and focusing of the image light by the optics block 130allows the electronic display 125 to be physically smaller, weigh lessand consume less power than larger displays. Additionally, magnificationmay increase the field of view of the content presented by theelectronic display 125. For example, the field of view of the displayedcontent is such that the displayed content is presented using almost all(e.g., approximately 110 degrees diagonal), and in some cases all, ofthe user's field of view. Additionally in some embodiments, the amountof magnification may be adjusted by adding or removing optical elements.

In some embodiments, the optics block 130 may be designed to correct oneor more types of optical error. Examples of optical error include barreldistortions, pincushion distortions, longitudinal chromatic aberrations,or transverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, comatic aberrations or errors dueto the lens field curvature, astigmatisms, or any other type of opticalerror. In some embodiments, content provided to the electronic display125 for display is pre-distorted, and the optics block 130 corrects thedistortion when it receives image light from the electronic display 125generated based on the content.

The IMU 140 is an electronic device that generates data indicating aposition of the AR headset 105 based on measurement signals receivedfrom one or more of the position sensors 135 and from depth informationreceived from the DCA 120. A position sensor 135 generates one or moremeasurement signals in response to motion of the AR headset 105.Examples of position sensors 135 include: one or more accelerometers,one or more gyroscopes, one or more magnetometers, another suitable typeof sensor that detects motion, a type of sensor used for errorcorrection of the IMU 140, or some combination thereof. The positionsensors 135 may be located external to the IMU 140, internal to the IMU140, or some combination thereof.

Based on the one or more measurement signals from one or more positionsensors 135, the IMU 140 generates data indicating an estimated currentposition of the AR headset 105 relative to an initial position of the ARheadset 105. For example, the position sensors 135 include multipleaccelerometers to measure translational motion (forward/back, up/down,left/right) and multiple gyroscopes to measure rotational motion (e.g.,pitch, yaw, roll). In some embodiments, the IMU 140 rapidly samples themeasurement signals and calculates the estimated current position of theAR headset 105 from the sampled data. For example, the IMU 140integrates the measurement signals received from the accelerometers overtime to estimate a velocity vector and integrates the velocity vectorover time to determine an estimated current position of a referencepoint on the AR headset 105. Alternatively, the IMU 140 provides thesampled measurement signals to the AR console 110, which interprets thedata to reduce error. The reference point is a point that may be used todescribe the position of the AR headset 105. The reference point maygenerally be defined as a point in space or a position related to the ARheadset's 105 orientation and position.

The IMU 140 receives one or more parameters from the AR console 110. Asfurther discussed below, the one or more parameters are used to maintaintracking of the AR headset 105. Based on a received parameter, the IMU140 may adjust one or more IMU parameters (e.g., sample rate). In someembodiments, certain parameters cause the IMU 140 to update an initialposition of the reference point so it corresponds to a next position ofthe reference point. Updating the initial position of the referencepoint as the next calibrated position of the reference point helpsreduce accumulated error associated with the current position estimatedthe IMU 140. The accumulated error, also referred to as drift error,causes the estimated position of the reference point to “drift” awayfrom the actual position of the reference point over time. In someembodiments of the AR headset 105, the IMU 140 may be a dedicatedhardware component. In other embodiments, the IMU 140 may be a softwarecomponent implemented in one or more processors.

The AR I/O interface 115 is a device that allows a user to send actionrequests and receive responses from the AR console 110. An actionrequest is a request to perform a particular action. For example, anaction request may be an instruction to start or end capture of image orvideo data or an instruction to perform a particular action within anapplication. The AR I/O interface 115 may include one or more inputdevices. Example input devices include: a keyboard, a mouse, a gamecontroller, or any other suitable device for receiving action requestsand communicating the action requests to the AR console 110. An actionrequest received by the AR I/O interface 115 is communicated to the ARconsole 110, which performs an action corresponding to the actionrequest. In some embodiments, the AR I/O interface 115 includes an IMU140, as further described above, that captures calibration dataindicating an estimated position of the AR I/O interface 115 relative toan initial position of the AR I/O interface 115. In some embodiments,the AR I/O interface 115 may provide haptic feedback to the user inaccordance with instructions received from the AR console 110. Forexample, haptic feedback is provided when an action request is received,or the AR console 110 communicates instructions to the AR I/O interface115 causing the AR I/O interface 115 to generate haptic feedback whenthe AR console 110 performs an action.

The AR console 110 provides content to the AR headset 105 for processingin accordance with information received from one or more of: the DCA120, the AR headset 105, and the AR I/O interface 115. In the exampleshown in FIG. 1, the AR console 110 includes an application store 150, atracking module 155 and an AR engine 145. Some embodiments of the ARconsole 110 have different modules or components than those described inconjunction with FIG. 1. Similarly, the functions further describedbelow may be distributed among components of the AR console 110 in adifferent manner than described in conjunction with FIG. 1.

The application store 150 stores one or more applications for executionby the AR console 110. An application is a group of instructions, thatwhen executed by a processor, generates content for presentation to theuser. Content generated by an application may be in response to inputsreceived from the user via movement of the AR headset 105 or the AR I/Ointerface 115. Examples of applications include: gaming applications,conferencing applications, video playback applications, or othersuitable applications.

The tracking module 155 calibrates the AR system environment 100 usingone or more calibration parameters and may adjust one or morecalibration parameters to reduce error in determination of the positionof the AR headset 105 or of the AR I/O interface 115. Calibrationperformed by the tracking module 155 also accounts for informationreceived from the IMU 140 in the AR headset 105 and/or an IMU 140included in the AR I/O interface 115. Additionally, if tracking of theAR headset 105 is lost (e.g., the DCA 120 loses line of sight of atleast a threshold number of SL elements), the tracking module 140 mayre-calibrate some or all of the AR system environment 100.

The tracking module 155 tracks movements of the AR headset 105 or of theAR I/O interface 115 using information from the DCA 120, the one or moreposition sensors 135, the IMU 140 or some combination thereof. Forexample, the tracking module 155 determines a position of a referencepoint of the AR headset 105 in a mapping of a local area based oninformation from the AR headset 105. The tracking module 155 may alsodetermine positions of the reference point of the AR headset 105 or areference point of the AR I/O interface 115 using data indicating aposition of the AR headset 105 from the IMU 140 or using data indicatinga position of the AR I/O interface 115 from an IMU 140 included in theAR I/O interface 115, respectively. Additionally, in some embodiments,the tracking module 155 may use portions of data indicating a positionor the AR headset 105 from the IMU 140 as well as representations of thelocal area from the DCA 120 to predict a future location of the ARheadset 105. The tracking module 155 provides the estimated or predictedfuture position of the AR headset 105 or the AR I/O interface 115 to theAR engine 145.

The AR engine 145 generates a 3D mapping of the area surrounding the ARheadset 105 (i.e., the “local area”) based on information received fromthe AR headset 105. In some embodiments, the AR engine 145 determinesdepth information for the 3D mapping of the local area based on timesfor light emitted by the DCA 120 to be detected by the DCA 120 afterbeing reflected by one or more objects in the area surrounding the ARheadset 105 or based on images of deformed SL elements captured by theDCA 120 of the AR headset 105. In various embodiments, the AR engine 145uses different types of information determined by the DCA 120 or acombination of types of information determined by the DCA 120.

The AR engine 145 also executes applications within the AR systemenvironment 100 and receives position information, accelerationinformation, velocity information, predicted future positions, or somecombination thereof, of the AR headset 105 from the tracking module 155.Based on the received information, the AR engine 145 determines contentto provide to the AR headset 105 for presentation to the user. Forexample, if the received information indicates that the user has lookedto the left, the AR engine 145 generates content for the AR headset 105that mirrors the user's movement in a virtual environment or in anenvironment augmenting the local area with additional content.Additionally, the AR engine 145 performs an action within an applicationexecuting on the AR console 110 in response to an action requestreceived from the AR I/O interface 115 and provides feedback to the userthat the action was performed. The provided feedback may be visual oraudible feedback via the AR headset 105 or haptic feedback via the ARI/O interface 115.

FIG. 2 is a wire diagram of one embodiment of an AR headset 200. The ARheadset 200 is an embodiment of the AR headset 105, and includes a frontrigid body 205, a band 210, a reference point 215, a left side 220A, atop side 220B, a right side 220C, a bottom side 220D, and a front side220E. The AR headset 200 shown in FIG. 2 also includes an embodiment ofa depth camera assembly (DCA) (not shown) which is further describedbelow in conjunction with FIGS. 3-5. The front rigid body 205 includesone or more electronic display elements of the electronic display 125(not shown), the IMU 130, an imaging aperture 225 and an illuminationaperture 230 to allow light to pass through, the one or more positionsensors 135, and the reference point 215. In some embodiments, portionsof the front rigid body 205 may be transparent (e.g., the front side220E, some other side, etc.) to visible light to allow a user to viewthe local area through the transparent portions of the front rigid body205.

FIG. 3 is a cross section of the front rigid body 205 of the AR headset200 depicted in FIG. 2. As shown in FIG. 3, the front rigid body 205includes the DCA 120 that includes an illumination source 304, animaging device 306, and a controller 308. Additionally, the front rigidbody 205 includes the electronic display 125, the optics block 130, theimaging aperture 225, and the illumination aperture 230, which arefurther described above in conjunction with FIGS. 1-2. The front rigidbody 205 also includes an exit pupil 335 where the user's eye 340 wouldbe located. In various embodiments, the illumination source 304 and theimaging device 306 are the front side 220E. In some embodiments, theimaging device 306 captures images of a local area 310, which is aportion of an environment surrounding the front rigid body 205 within afield of view of the imaging device 306. For purposes of illustration,FIG. 3 shows a cross section of the front rigid body 205 in accordancewith a single eye 340. In some embodiments, the DCA 120 or somecomponents of the DCA are in line with the user's eye 340. In otherembodiments, the components of the DCA are not in line with the user'seye 340.

The electronic display 125 emits light forming an image toward theoptics block 130, which alters the light received from the electronicdisplay 125. The optics block 130 directs the altered image light to theexit pupil 335, which is a location of the front rigid body 205 where auser's eye 340 is positioned. FIG. 3 shows a cross section of the frontrigid body 205 for a single eye 340 of the user, with another electronicdisplay 125 and optics block 130, separate from those shown in FIG. 3,included in the front rigid body 205 to present content, such as anaugmented representation of the local area 310 or virtual content, toanother eye of the user.

The DCA 120 includes the illumination source 304, the imaging device306, and the controller 308. The illumination source 304 generatespulses of light used to determine depth information associated with thelocal area 310. The illumination source 304 generates one or more lightpulses in response to receiving an instruction including an illuminationparameter from the controller 308. An illumination parameter is aninstruction used by an illumination source to generate one or more lightpulses. An illumination parameter may be, e.g., a trigger signal to sendone or more light pulses, a pulse width, a pulse amplitude, a pulseperiod, a duty cycle, a quantity of pulses to emit, a pulse wavelength,or some combination thereof. A light pulse may be a single pulse or atrain of pulses (also referred to as a comb pulse) that include a seriesof pulses that have a specific period.

The illumination source 304 generates one or more light pulses in one ormore bands such as a visible band (˜380 nm to 750 nm), an infrared (IR)band (˜750 nm to 1500 nm), an ultraviolet band (10 nm to 380 nm),another portion of the electromagnetic spectrum, or some combinationthereof. The generated light may be coherent, semi-coherent, and/orincoherent. The illumination source 304 may generate light pulses at aframe rate (e.g., 120 Hz, 240 Hz, etc.) of the imaging device 306. Insituations where a series of pulses is expected for a given frame at theimaging device 306, the illumination source 304 may generate lightpulses significantly faster than the frame rate (e.g. 100, 10,000 orupwards times the frame-rate). Additionally, there is low jitter (on theorder of several nanoseconds or less) in the latency time betweentriggering the illumination source 304 to generate a pulse (or series ofpulses) and the illumination source 304 generating the pulse (or seriesof pulses), and low error in the measured latency. The illuminationsource 304 projects the one or more light pulses out of the illuminationaperture 230 onto the local area 310.

The imaging device 306 generates image data of the local area 310through the imaging aperture 225. The imaging device 306 captures imagedata responsive to receiving an instruction including an exposureparameter from the controller 308. An exposure parameter is aninstruction used by an imaging device to generate image data. Anexposure parameter may be, e.g., a trigger signal to record lightthereby to generate image data, an exposure time period to record lightthereby to generate image data, a start time for an exposure timeperiod, or a start time and length of exposure for each pulse in theexpected series, or some combination thereof.

The imaging device 306 detects and conveys the information carried inthe light it captures into signals. The imaging device 306 is configuredto capture images at least in the same spectrum as the illuminationsource 304. The imaging device 306 may also be sensitive to visiblelight, IR light, or light having wavelengths in other portions of theelectromagnetic spectrum. In some embodiments, the imaging device 306 isa camera and comprises a charge coupled device (CCD), a complementarymetal-oxide-semiconductor (CMOS) imager, other light sensitive device,or some combination thereof. Additionally, there is low jitter (on theorder of several nanoseconds) between triggering the imaging device 306to start an exposure (i.e., capture an image) and the imaging device 306starting the exposure. Similarly, there is low jitter (on the order ofseveral nanoseconds) between triggering the imaging device 306 to stopan exposure (i.e., stop the capture of an image) and the imaging device306 stopping the exposure. In addition, knowledge of the actual startand stop time would be accurate to a fraction of the jitter value (e.g.,below a nanosecond).

The controller 308 regulates the illumination source 304 usingillumination parameters. The controller 308 provides one or moreinstructions including one or more illumination parameters to theillumination source 304 that emits light pulse(s) in accordance with theone or more illumination parameters. The controller 308 determines theone or more illumination parameters such that between each light pulsethere is at least a time period that allows for a light pulse to clearthe local area. For example, if the local area 310 is 20 feet long, thena time between pulses would be at least 40 nanoseconds—as light travelsat approximately 1 foot-per-nanosecond. The controller 308 may determinethe frequency of light pulses emitted by the illumination source 304based on a size of the local area 310. In addition, the controller 308can set the pulse width (on the order or nanoseconds) for each emittedpulse, and the specific amplitude of the pulse (emitted power) on apulse-by-pulse or frame-by-frame basis. In some embodiments, anillumination parameter may be a trigger and at least one pulsecharacteristic of a light pulse emitted by the illumination source 304is determined at the illumination source 304. For instance, thecontroller 308 may regulate the illumination source 304 to emit a lightpulse having a specific pulse width (e.g., 6 ns), however, the pulseamplitude is determined by illumination source 304.

The controller 308 regulates the imaging device 306 to capture reflectedlight to generate image data of the local area 310. The controller 308provides one or more instructions including one or more exposureparameters (e.g., an exposure time period to capture reflected light, astart time for an exposure time period, and repetition frequency/countfor every read-out frame, etc.) to the imaging device 306. As such, thecontroller 308 regulates an exposure start time, an exposure duration,and repeat count to the imaging device 306. During the exposure timeperiod the imaging device 306 records light and converts light intoimage data. The controller 308 may determine the exposure start timeaccording to a distance to a depth zone from the DCA 120. The size of adepth zone depends on the length of the exposure time period.Accordingly, the controller 308 may determine the exposure time periodaccording a size of the depth zone. The location of the depth zone isdetermined by the start time of the exposure period relative to the timethe pulse was emitted from the illumination source 304. Accordingly, thecontroller 308 may select which depth zone to collect image data on byadjusting the start time of the exposure period relative to the time thepulse was emitted from the illumination source 304. For example, thecontroller 308 determines an exposure start time based on a distanceD_(near) of a near border (border of a depth zone closer to DCA 120) ofthe depth zone to the front plane 322. A time difference between thestart time and the time point when the illumination source 304 istriggered to emit a light pulse equals to twice D_(near) divided by aspeed of light (e.g., ˜3×10⁸ meters/second). As such, the controller 308regulates the imaging device 306 to capture light reflected off allobjects of the depth zone.

Objects included in the local area 310 reflect incident ambient light aswell as light emitted by the illumination source 304 onto the local area310. The local area 310 includes one or more depth zones at differentdistances from the front plane 322. A depth zone includes objects in thelocal area that are within a predetermined distance range from the frontplane 322. For example, the local area 310 includes a near-depth zone350, a mid-depth zone 352, and a far-depth zone 354. The near-depth zone350 includes objects that are within a first predetermined distancerange (e.g., less than one meter) from the front plane 322. Themid-depth zone 352 includes objects that are within a secondpredetermined distance range (e.g., between one meter and two meters)from the front plane 322. The far-depth zone 354 includes objects thatare within a third predetermined distance range (e.g., greater than twometers) from the front plane 322. One of ordinary skill in the art wouldappreciate that the local area 310 may include any number of depth zonesof different sizes, each of which may include objects within apredetermined distance range.

In some embodiments, the controller 308 divides the local area 310 intoa fixed number of depth zones and identifies objects in each depth zone.For example, the controller 308 divides the local area 310 into threedepth zones: the near-depth zone 350, the mid-depth zone 352, and thefar-depth zone 354. The controller 308 locates objects of the local area310 in each depth zone. For example, the controller 308 regulates theillumination source 304 to emit one or more light pulses for each depthzone, and regulates the imaging device 306 to capture respectivereflected light thereby to generate image data. For a given depth zone,the controller 308 may analyze the image data to identify objects in thegiven depth zone. The controller 308 associates the identified objectswith a depth information attributed to the depth zone they are locatedin. For example, a distance of an object to the DCA 120 is based on atime difference between the illumination source 304 emitting a lightpulse and a start time of the exposure period. As light travels at afixed speed in air (e.g., 3×10⁸ meters/second), the distance to a depthzone is half of the interval light travels during the time difference.Accordingly, the controller 308 may attribute depth information toobjects identified in a given depth zone.

In some embodiments, the controller 308 may automatically divide thelocal area 310 into a relatively large number of depth zones (e.g., eachcorresponding to an inch) and generate image data for each of the depthzones. The controller 308 identifies which depth zones include objectsand generates a new set of depth zones that emphasize the identifiedobjects, and not the empty space between the identified objects. Forexample, areas that include objects may be associated with a largenumber of depth zones, however, empty space between objects may beassociated with low number of depth zones (e.g., a single depth zone).

In alternate embodiments, the controller 308 divides the local area 310into a relatively low number of large depth zones (e.g., 3). Thecontroller 308 identifies which depth zones include objects, and dividesthe identified depth zones into smaller depth zones. The controller 308again identifies which of the smaller depth zones include objects, andfurther divides the identified smaller depth zones into even smallerdepth zones. The controller 308 continues this process until somethreshold level of depth resolution is obtained.

The controller 308 may further adjust a size of a depth zone based onthe objects located in that depth zone. The size of the depth zone isdetermined by a near border (i.e., a border having the shortest distanceto the front plane 322) and a far border (i.e., a border having thelongest distance to the front plane 322.) For example, the controller308 adjusts the near (far) border of a depth zone to the location of theobject having the shortest (longest) distance among all objectsidentified in the depth zone.

The controller 308 may further determine an object's reflectivity basedon the light reflected off the object and captured by the imaging device306. A higher reflectivity object is brighter than a lower reflectivityobject under the same or similar irradiance and distance. For example,for an object, the controller 308 generates an image of the object andcompares the received signal of the image of the object to a thresholdvalue associated with the distance of the object. When the brightness isgreater (less) than the threshold value, the controller 308 determinesthe reflectivity of the object is high (low). In addition, throughtemporal data collection, the estimation of reflectivity, and moreaccurately the object Bi-Directional Reflectance Distribution Function(BRDF), can be continuously refined to support tracking, mapping, andensuring the emitted illumination signal is within an acceptable rangefor the objects characteristics. In addition, the controller 308 maydetermine movement of an object relative to the DCA 120. For example,the controller 308 identifies changes in locations of an object from oneimage to another image to determine movement of the object. Thecontroller 308 compares a speed of movement of the object to a thresholdspeed. When the speed is greater (less) than the threshold speed, thecontroller 308 determines the movement of the object is high (low).

The controller 308 determines one or more illumination parameters basedon a depth zone, object(s) included in the depth zone, an imageresolution, and/or image quality. The image resolution may be configuredby a user. For example, the controller 308 regulates the illuminationsource 304 to emit light pulses having a lower (higher) amplitude tocapture images of the near-depth zone 350 (the far-depth zone 354).Because the intensity of a light beam attenuates over time when ittravels through various volumes of materials in the local area 310,objects that are more distant from the front plane 322 are lit by lightof less intensity than those closer to the front plane 322. In addition,when working within a single identified depth zone, an estimate of theambient illumination properties in the other (not active illuminated)depth zones can be determined. This supports ambient subtraction methodswhen subsequent image frames provide active illumination in thoserespective depth zones.

As another example, the controller 308 regulates the illumination source304 to emit light pulses having a lower (higher) amplitude to captureimages of a depth zone of which the object(s) included have high (low)reflectivity. As a further example, the controller 308 regulates theillumination source 304 to emit a light pulse of which the quantity ofsingle pulses is lower (higher) to capture images of a depth zone ofwhich the movement of the object(s) is low (high). The controller 308regulates the illumination source 304 to emit a light pulse of which therepeat frequency is lower (higher) to capture images of a depth zone ofwhich the movement of the object(s) is low (high). Images of differentobjects included in a depth zone may have different brightness due tothe distances and/or reflectivity of the objects under the sameillumination. Images of the bright object(s) saturate when thedifferences in brightness exceed the dynamic range of the imaging device306. As yet a further example, the controller 308 may regulate theillumination source 304 to emit a light pulse of which the quantity ofsingle pulses is lower (higher) to capture images of lower (higher)resolution.

The controller 308 may further determine the one or more illuminationparameters to ensure image quality. The controller 308 analyzes imagequality (e.g., noise, contrast, sharpness, etc.) of the captured images.For example, the controller 308 analyzes the captured images todetermine a noise measure (e.g., Gaussian noise, salt and pepper noise,shot noise, quantization noise, film grain, etc.) and determines theillumination parameter(s) to ensure that the noise measure is below athreshold. As another example, the controller determines a signal tonoise ratio (SNR) of the measured image and determine the illuminationparameter(s) to ensure that the SNR is above a threshold. As a furtherexample, the controller 308 analyses the captured images to determine acontrast measure (e.g., luminance contrast, color contrast) anddetermines the illumination parameter(s) to ensure that the contrastmeasure is above a threshold. The controller 308 may further determinethe illumination parameter(s) to maximize the image quality of thecaptured images such as to minimize a noise measure, to maximize a SNR,and/or to maximize a contrast measure of a captured image. To improve orto enhance image quality of captured images, the controller 308regulates the illumination source 304 to emit a light pulse that has ahigher amplitude, that has a higher quantity of single pulses, or thathas a higher repeat frequency.

The controller 308 monitors the identified depth zones, for example, byanalyzing image data generated for the depth zones. Because objects maymove, the controller 308 monitors the identified objects and adjusts thedetermined depth zones. For example, responsive to determining that anobject has moved, the controller 308 determines an updated distance ofthe object and uses the updated distance to adjust one or more depthzones. For instance, the controller 308 may adjust the near borderand/or the far border a depth zone to which the identified object belongthrough determining that the object becomes the closest or furthestobject of the depth zone. The controller 308 may further create a newdepth zone when an object moves into a predetermined distance range ofthe new depth zone. The controller 308 may remove a depth zone when thelast object moves outside the predetermined distance range of the depthzone.

The controller 308 may further adjust regulating the imaging device 306and the illumination source 304 based on the adjusted depth zones tocapture images of the depth zones. That is, the controller 308 mayadjust one or more illumination parameters and/or one or more exposureparameters to capture images of the depth zones. In this manner,whenever the illumination source 304 is activated, it can send a singleor a series of pulses into any predefined or determined range ofinterest for the imaging device 306 to synchronize the integrationwindow(s). This allows a finer control on the received signal (to bewithin the prescribed dynamic range of the imaging device 306), whilestill utilizing an affordable pulsed led or laser technology that isalso eye-safe to users. In turn, the imaging device 306 can synchronizethe integration windows appropriate to the range or ranges of interestin any given frame, and record/collect light at times associated witheach pulse associated with each of the range or ranges of interest. Thissupports charge summation during any given pulse window multiple timesfor any frame, all in the analog domain of the detector (in current CCDor CMOS architectures), which minimizes any potential digital or readnoise artifacts and increases the signal of interest by not integratingambient signals (noise source) between pulses.

Each image or image data can be associated with a particular depth zone.Image data may include one or more identified objects. For a given imagedata associated with a depth zone, the controller 308 determines thepixel values in the given image data that correspond to objects in theassociated depth zone. The controller 308 associates depth informationfor the depth zone to the pixel values corresponding to the depth zoneto generate a partially augmented image. A partially augmented imageincludes image data for some, but not all, of the depth zones in a localarea. The controller 30 may generate a partially augmented image foreach of the depth zones (e.g., near 350, mid 352, and far 354) in thelocal area 310, and may combine the partially augmented images into anaugmented image that includes depth information for the imaged portionof the local area. In some embodiments, the augmented image and/or apartially augmented image may be combined with an RGB image of the localarea 310. Image data is used to generate partially augmented images, andin some embodiments, partially augmented images and/or augmented imagesinclude metadata describing for e.g., illumination parameters and/orexposure parameters used to generate the image data. In alternateembodiments, the controller 308 generates augmented images of the localarea 310 by integrating the partially augmented images (for all of thedepth zones) while maintaining the depth information associated witheach pixel. The controller 308 provides augmented images to theaugmented reality console 110.

FIG. 4 illustrates an example DCA 120 capturing image data of a localarea including one or more depth zones. The DCA 120 scans the local areaand identifies a first near object 404, a second near object 406, and afar object 408. The DCA 120 also determines the location and thedistance of each object from the DCA 120. In addition, the DCA 120 maydetermine the reflectivity and/or the movement of each object relativeto the user. Based on, for example, the distance of the identifiedobjects, the DCA 120 divides the scene into depth zones: (1) anear-depth zone 402 including the first near-distance object 404 and thesecond near-distance object 406, and (2) a far-depth zone 403 includingthe far-distance object 408.

The DCA 120 captures an image of a depth zone at one time and capturesimages of different depth zones at different times. At t₁, the DCA 120emits a single light pulse 414 to capture an image of the near-depthzone 402. The DCA 120 may determine one or more illumination parametersof the light pulse 414 based on the near-depth zone 402, the first andsecond near objects 404 and 406, and/or the image resolution. Forexample, the DCA 120 determines that the light pulse 414 includes onesingle light pulse and has a pulse width of t₁-t₂. The DCA 120determines one or more exposure parameters to generate image data fromthe near-depth zone 402. Based on a near border 405 of the near-depthzone 402, the DCA 120 determines t₃ as an exposure start time andregulates its imaging device to capture the reflected light pulse 424 att₃. The reflected light pulse 424 is the light pulse 414 being reflectedoff the near-distance object 404. A time period from t₁ to t₃ is thetime period it takes for the light pulse 414 to travel from the DCA 120to the near border 405 of the near-depth zone 402 and back to the DCA120. That is, the one half of the time period from t₁ to t₃ correspondsto a distance to the near border 405 from the DCA 120. In addition, theDCA 120 determines the exposure duration. In capturing image data forthe near depth zone 402, the exposure duration corresponds to thedistance between the near border 405 and a far border 407. The reflectedlight pulse 426 is the light pulse 414 being reflected off the nearobject 406. The time period from t₃ to t₄ is the time period between afirst time point when the rising edge of the light pulse 414 arrives atthe near border 405 and a second time point when the falling edge of thelight pulse 414 arrives at the far border 407.

For the near depth zone 402, the DCA 120 generates image data using thedetermined exposure start time and exposure time period. The near depthzone 402 is associated with depth information corresponding to adistance to the near depth zone 402 (or some point within the near depthzone 402). The DCA 120 generates a partially augmented image by mappingpixel values of objects in the image data to the depth informationassociated with the near depth zone 402.

The DCA 120 then shifts focus toward the far depth zone 403. At t₅, theDCA 120 emits a single light pulse 418 to capture an image of thefar-depth zone 403. The DCA 120 may determine one or more illuminationparameters of the light pulse 418 based on the far-depth zone 403, thefar object 408, and/or the image resolution. For example, the DCA 120determines that the light pulse 418 includes one single light pulse andhas a pulse width of t₅-t₆. The DCA 120 determines one or more exposureparameters to generate image data from the far-depth zone 403. Based ona near border 409 of the far-depth zone 403, the DCA 120 determines t₇as an exposure start time and regulates its imaging device to capture areflected light pulse 428 at t₇. The reflected light pulse 428 is thelight pulse 418 being reflected off the far object 408. A time periodfrom t₅ to t₇ is the time period it takes for the light pulse 418 totravel from the DCA 120 to the far border 410 of the far-depth zone 403and back to the DCA 120. In addition, the DCA 120 determines theexposure duration. In capturing image data for the far depth zone 403,the exposure duration corresponds to the distance between the nearborder 409 and a far border 410. The time period t₇-t₈ is the timeperiod between a first time point when the rising edge of the lightpulse 418 arrives at the near border 409 and a second time point whenthe falling edge of the light pulse 418 arrives at the far border 410.

For the far depth zone 403, the DCA 120 generates image data using thedetermined exposure start time and exposure time period. The far depthzone 403 is associated with depth information corresponding to adistance to the far depth zone 403 (or some point within the far depthzone 403). The DCA 120 generates another partially augmented image bymapping pixel values of objects in the image data to the depthinformation associated with the far depth zone 403.

The DCA 120 generates an augmented image using both of the partiallyaugmented images. The DCA 120 combines the partially augmented imagesinto an augmented image that includes depth information for the imagedportion of the local area (i.e., the near depth zone 402 and the fardepth zone 403). In some embodiments, the augmented image and/or apartially augmented image may be combined with an RGB image of the localarea 310. The DCA 120 may further operate with a spatially modulatedsignal for the illumination source, which provides for a structuredlight depth retrieval method (utilize triangulation, or a known baselinedistance between the illuminator and camera to determine a depthcalculation). This allows the system to work with both temporally(pulsed light, either single or multiple per frame as described) andspatially modulated light sources to refine the depth retrievalestimation.

Note, that in the above example, for a given depth zone, a single pulseis primarily used to generate a partially augmented image. However, inalternate embodiments, multiple pulses are emitted for a given depthzone before reading out image data associated with the depth zone. Thisimproves a signal strength associated with image data in the given depthzone, as light from successive pulses are added in a near noiselessfashion. The DCA 120 can then read out image data associated for thedepth zone that was generated using a plurality of pulses (e.g., 100pulses). The improved signal strength is useful in capturing accurateimage data for objects with low reflectivity and/or objects that aredistant from the DCA 120.

FIG. 5 illustrates a flow diagram of a process of generating augmentedimages, according to an embodiment. The process of FIG. 5 may beperformed by the DCA 120. Other entities may perform some or all of thesteps of the process in other embodiments. Likewise, embodiments mayinclude different and/or additional steps, or perform the steps indifferent orders.

A DCA determines 502 one or more depth zones in a local area. Each depthzone has different depth information (e.g., object(s) included and therespective distance to the DCA 120). The DCA identifies objects includedin the local area and determines a distance of each identified object tothe DCA. The DCA further divides the local area into one or more depthzones based on the determined distance of the identified objects.

The DCA illuminates 504 a depth zone, of the one or more depth zones,that includes one or more objects. The DCA may regulate its illuminationsource to emit a light pulse to illuminate the depth zone. The DCA maydetermine one or more illumination parameters (e.g., a pulse width, anamplitude, a frequency, a quantity of pulses included, and/or awavelength) and regulate its illumination source according to thedetermined illumination parameters. The DCA may determine one or moreillumination parameters based on the depth zone, object(s) included inthe depth zone, and/or an image resolution. In one embodiment, the DCAemits a light pulse of which the width is 6 ns or shorter.

The DCA captures 506 light reflected off objects in the depth zone togenerate image data of the depth zone. The DCA regulates its imagingdevice to capture light reflected off the objects in the depth zoneresponsive to being illuminated by the light pulse emitted by itsillumination source. The DCA may determine one or more exposureparameters (e.g., start time, duration, etc.) and regulate its imagingdevice according to the determined illumination parameters. The DCA maydetermine one or more exposure parameters based on the depth zone and/orthe corresponding light pulse emitted. The generated image data includepixel values (e.g., RGB values) for pixels of an image of the depthzone. The DCA may monitor the depth zone based on the generated imagedata. The DCA may adjust one or more illumination parameters, one ormore exposure parameters, a near border of the depth zone, and/or a farborder of the depth zone and adjusts the illumination of a depth zoneaccordingly.

The DCA determines 506 whether it has generated image data for allidentified depth zones, for example, within the DCA's range or basedupon its sampling basis and knowledge from previous data sets on objectsof interest locations. If image data for all identified depth zones hasnot been generated, the process flow moves to 504, and a different depthzone is illuminated.

Responsive to determining that image data has been generated for allidentified depth zones, the DCA generates 508 an augmented image of thedepth zone based on the image data and the depth information for allidentified depth zones. The DCA determines partially augmented imagesfor the image data associated with different depth zones, and combinesthe partially augmented image data into an augmented image. The DCAprovides 510 the augmented image of the local area to an AR console forfurther processing.

Additional Configuration Information

The foregoing description of the embodiments of the disclosure has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of thedisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the disclosure may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that isproduced by a computing process described herein. Such a product maycomprise information resulting from a computing process, where theinformation is stored on a non-transitory, tangible computer readablestorage medium and may include any embodiment of a computer programproduct or other data combination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the disclosure be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thedisclosure, which is set forth in the following claims.

What is claimed is:
 1. A range-gated depth camera assembly, comprising:an illumination source configured to emit illumination light pulses; animaging device configured to capture a plurality of images of a localarea; and a controller configured to: responsive to a determination thata contrast measure of an image of the plurality of images is below acontrast threshold value, adjust one or more illumination parameters,the one or more illumination parameters comprising at least an amplitudeof the light pulses and a quantity of the light pulses; regulate theillumination source to emit a plurality of illumination light pulses toilluminate a scene of the local area according to the one or moreillumination parameters, the local area being divided into a pluralityof depth zones comprising at least one non-empty depth zone, thenon-empty depth zone comprising at least one object; for each non-emptydepth zone of the plurality of depth zones, regulate the imaging deviceto record a set of reflected light pulses to generate image data for thenon-empty depth zone, the set of reflected light pulses being theillumination light pulses reflected off one or more objects included inthe non-empty depth zone; and generate one or more partially augmentedimages of the scene using the image data generated for the at least onenon-empty depth zone of the plurality of non-empty depth zones, whereinat least one of the one or more partially augmented images include depthinformation for the at least one object, and the one or more partiallyaugmented images are used to generate virtual content for presentationto a user.
 2. The range-gated depth camera assembly of claim 1, whereina first exposure time period for a first depth zone of the depth zonesis different from a second exposure time period for a second depth zoneof the depth zones, the first depth zone different from the second depthzone.
 3. The range-gated depth camera assembly of claim 1, whereingenerating the one or more partially augmented images of the scenecomprises associating the image data of a depth zone with depthinformation of the depth zone.
 4. The range-gated depth camera assemblyof claim 1, wherein the controller is further configured to adjust theone or more illumination parameters based at least on one of a distancefrom the imaging device to one depth zone of the depth zones and aquality of the image of the plurality of images.
 5. The range-gateddepth camera assembly of claim 1, wherein the controller is furtherconfigured to adjust the one or more illumination parameters based on atleast one of a degree of reflectivity of an object, a speed of motion ofthe object, a distance of the object from the imaging device, a noisemeasure of the image, and a signal to noise ratio (SNR) of the image,the one or more illumination parameters comprising at least one of apulse width of the light pulses, an amplitude of the light pulses, afrequency of the light pulses, a quantity of the light pulses, and awavelength of the light pulses.
 6. The range-gated depth camera assemblyof claim 1, wherein the controller is further configured to regulate theimaging device according to one or more exposure parameters and thecontroller is further configured to: determine updated one or moreexposure parameters based at least on an illumination parameter of theillumination source and characteristics of the depth zones; and adjustthe one or more exposure parameters according to the updated one or moreexposure parameters, the one or more exposure parameters comprising atleast one of an exposure time period, a start time for the exposure timeperiod, a repetition frequency for every read-out frame, and arepetition count for every read-out frame.
 7. The range-gated depthcamera assembly of claim 1, wherein the controller is further configuredto: divide the local area into the plurality of depth zones, wherein anumber of the plurality of depth zones is predetermined; and for eachnon-empty depth zone of the plurality of depth zones, locate the one ormore objects within a predetermined distance range from the imagingdevice to identify the one or more objects.
 8. The range-gated depthcamera assembly of claim 1, wherein the controller is further configuredto: divide the local area into a second plurality of predetermined depthzones, a dimension of each predetermined depth zone being predetermined,identify a plurality of empty depth zones, the plurality of empty depthzones being a subset of the second plurality of predetermined depthzones that include no object, and combine two of the identifiedplurality of empty depth zones that are adjacent to each other into asingle empty depth zone.
 9. The range-gated depth camera assembly ofclaim 1, wherein the controller is further configured to: divide thelocal area into a first set of depth zones, identify a set of nonemptydepth zones, the set of nonempty depth zones being a subset of the firstset of depth zones that include at least one object, and for eachnonempty depth zone, divide the nonempty depth zone into a second set ofdepth zones, a dimension of each depth zone of the second set of depthzones greater than a threshold dimension.
 10. The range-gated depthcamera assembly of claim 1, wherein the controller is further configuredto: for each depth zone of the plurality of depth zones: adjust a firstborder of the depth zone to a first location of a first object, thefirst object having a shortest distance to the imaging device among theone or more objects included in the depth zone, and adjust a secondborder of the depth zone to a second location of a second object, thesecond object having a longest distance to the image device among theone or more objects included in the depth zone.
 11. The range-gateddepth camera assembly of claim 1, wherein the controller is furtherconfigured to: detect movement of an object included in a depth zone,responsive to detecting movement of the object, determine an updateddistance from the object to the imaging device, and generate a secondplurality of adjusted depth zones by adjusting the plurality of depthzones based on the updated distance, the adjusting comprising at leastone of adjusting a border of a depth zone of the plurality of depthzones, creating a new depth zone responsive to determining that theupdated distance is within a first predetermined range, and removing anexisting depth zone responsive to determining that the updated distanceis outside a second predetermined range.
 12. A head-mounted display(HMD) comprising: an electronic display; a range-gated depth cameraassembly, comprising: an illumination source configured to emitillumination light pulses, and an imaging device configured to capture aplurality of images of a local area; and a controller configured to:responsive to a determination that a contrast measure of an image of theplurality of images is below a contrast threshold value, adjust one ormore illumination parameters, the one or more illumination parameterscomprising at least an amplitude of the light pulses and a quantity ofthe light pulses; regulate the illumination source to emit a pluralityof illumination light pulses to illuminate a scene of the local areaaccording to the one or more illumination parameters, the local areabeing divided into a plurality of depth zones comprising at least onenon-empty depth zone, the non-empty depth zone comprising at least oneobject; for each non-empty depth zone of the plurality of depth zones,regulate the imaging device to record a set of reflected light pulses togenerate image data for the non-empty depth zone, the set of reflectedlight pulses being the illumination light pulses reflected off one ormore objects included in the non-empty depth zone; and generate one ormore partially augmented images of the scene using the image datagenerated for the at least one non-empty depth zone of the plurality ofnon-empty depth zones; wherein at least one of the one or more partiallyaugmented images include depth information for the at least one object,and the one or more partially augmented images are used to generatevirtual content for presentation to a user.
 13. The HMD of claim 12,wherein a first exposure time period for a first depth zone of the depthzones is different from a second exposure time period for a second depthzone of the depth zones, the first depth zone different from the seconddepth zone.
 14. The HMD of claim 12, wherein generating the one or morepartially augmented images of the scene comprises associating the imagedata of a depth zone with depth information of the depth zone.
 15. TheHMD of claim 12, wherein the controller is further configured to adjustthe one or more illumination parameters based at least on one of adistance from the imaging device to one depth zone of the depth zonesand a quality of the image of the plurality of images.
 16. The HMD ofclaim 12, wherein the controller is further configured to adjust the oneor more illumination parameters based on at least one of a degree ofreflectivity of an object, a speed of motion of the object, a distanceof the object from the imaging device, a noise measure of the image, anda signal to noise ratio (SNR) of the image, the one or more illuminationparameters comprising at least one of a pulse width of the light pulses,an amplitude of the light pulses, a frequency of the light pulses, aquantity of the light pulses, and a wavelength of the light pulses. 17.The HMD of claim 12, wherein the controller is further configured toregulate the imaging device according to one or more exposure parametersand the controller is further configured to: determine updated one ormore exposure parameters based at least on an illumination parameter ofthe illumination source and characteristics of the depth zones; andadjust the one or more exposure parameters according to the updated oneor more exposure parameters, the one or more exposure parameterscomprising at least one of an exposure time period, a start time for theexposure time period, a repetition frequency for every read-out frame,and a repetition count for every read-out frame.
 18. A methodcomprising: determining a contrast measure of an image of a plurality ofimages of a local area is below a contrast threshold value, theplurality of images captured by an imaging device; adjusting one or moreillumination parameters based on the contrast measure being below thecontrast threshold value, the one or more illumination parameterscomprising at least an amplitude of a plurality of light pulses and aquantity of the plurality of light pulses; emitting, by an illuminationsource, the plurality of illumination light pulses to illuminate a sceneof the local area according to the one or more illumination parameters,the local area being divided into a plurality of depth zones comprisingat least one non-empty depth zone, the non-empty depth zone comprisingat least one object; for each depth zone of the plurality of depthzones, recording a set of reflected light pulses to generate image datafor the non-empty depth zone, the set of reflected light pulses beingthe illumination light pulses reflected off one or more objects includedin the non-empty depth zone; and generating one or more partiallyaugmented images of the scene using the image data generated for the atleast one non-empty depth zone of the plurality of non-empty depthzones, wherein at least one of the one or more partially augmentedimages include depth information for the at least one object, and theone or more partially augmented images are used to generate virtualcontent for presentation to a user.
 19. The method of claim 18, whereina first exposure time period for a first depth zone of the depth zonesis different from a second exposure time period for a second depth zoneof the depth zones, the first depth zone different from the second depthzone.
 20. The method of claim 18, wherein generating the one or morepartially augmented images of the scene comprises associating the imagedata of a depth zone with depth information of the depth zone.